A Combined Neural Network/Gaussian Process Regression Time Series Forecasting System for the NN3 Competition

In a recent study [1] we have conducted a large scale empirical comparison of seven different machine learning models for time series forecasting using the M3 benchmark data. The outcome of this study is that the standard multilayer perceptron neural network (MLP) and Gaussian process regression (GPR) have turned out to be respectively the first and the second best methods. Taking cue from this study, we propose here a combined model that applies MLP and GPR using a number of different input preprocessing methods. The model combination includes some aspects of forecast combination using simple averaging and model selection based on the training set and validation set performance. What makes this competition challenging is the large forecast horizon (the next eighteen points have to be forecasted). The study in [1] was simply for the one-step ahead case. We have devised input preprocessing steps that seem to be suitable for different forecast horizon ranges. In addition to the machine learning part, we have carried out a thorough preprocessing of the time series, including deseasonalization (if needed), log transformation, and scaling. Next section gives a description of the MLP model and the GPR model,